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LOW-COMPLEXITY FILM GRAIN SIMULATION TECHNIQUE 

CROSS-REFERENCE TO RELATED APPLICATIONS 

5 This application claims priority under 35 U.S.C. 119(e) to U.S. Provisional Patent 

Application Serial No 60/630640, filed November 23, 2004, the teachings of which are 
incorporated herein. 

TECHNICAL FIELD 

10 

This invention relates to a technique for simulating film grain in an image. 
BACKGROUND OF THE INVENTION 

1 5 Motion picture films comprise silver-halide crystals dispersed in an emulsion, which is 

coated in thin layers on a film base. The exposure and development of these crystals form the 
photographic image consisting of discrete tiny particles of silver. In color negatives, the silver 
undergoes chemical removal after development and tiny blobs of dye occur on the sites where 
the silver crystals form. These small specks of dye are commonly called 'grain' in color film. 

20 Grain appears randomly distributed on the resulting image because of the random formation 
of silver crystals on the original emulsion. Within a uniformly exposed area, some crystals 
develop after exposure while others do not. 

Grain varies in size and shape. The faster the film, the larger the clumps of silver 
formed and blobs of dye generated, and the more they tend to group together in random 

25 patterns. The grain pattern is typically known as 'granularity'. The naked eye cannot 

distinguish individual grains, which vary from 0.0002 mm to about 0.002 mm. Instead, the 
eye resolves groups of grains, referred to as blobs. A viewer identifies these groups of blobs 
as film grain. As the image resolution becomes larger, the perception of the film grain 
becomes higher. Film grain becomes clearly noticeable on cinema and high-definition 

30 images, whereas film grain progressively loses importance in SDTV and becomes 
imperceptible in smaller formats. 

Motion picture film typically contains image-dependent noise resulting either from the 
physical process of exposure and development of the photographic film or from the 
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subsequent editing of the images. The photographic film possesses a characteristic quasi- 
random pattern, or texture, resulting from physical granularity of the photographic emulsion. 
Alternatively, a similar pattern can be simulated over computed-generated images in order to 
blend them with photographic film. In both cases, this image-dependent noise is referred to as 
5 grain. Quite often, moderate grain texture presents a desirable feature in motion pictures. In 
some instances, the film grain provides visual cues that facilitate the correct perception of 
two-dimensional pictures. Film grain is often varied within a single film to provide various 
clues as to time reference, point of view, etc. Many other technical and artistic uses exist for 
controlling grain texture in the motion picture industry. Therefore, preserving the grainy 

10 appearance of images throughout image processing and delivery chain has become a 
requirement in the motion picture industry. 

Several commercially available products have the capability of simulating film grain, 
often for blending a computer-generated object into a natural scene. Cineon® from Eastman 
Kodak Co, Rochester New York, one of the first digital film applications to implement grain 

15 simulation, produces very realistic results for many grain types. However, the Cineon® 
application does not yield good performance for many high speed films because of the 
noticeable diagonal stripes the application produces for high grain size settings. Further, the 
Cineon® application fails to simulate grain with adequate fidelity when images are subject to 
previous processing, for example, such as when the images are copied or digitally processed. 

20 Another commercial product that simulates film grain is Grain Surgery™ from Visual 

Infinity Inc., which is used as a plug-in of Adobe ® After Effects ®. The Grain Surgery™ 
product appears to generate synthetic grain by filtering a set of random numbers. This 
approach suffers from disadvantage of a high computational complexity. 

None of these past schemes solves the problem of restoring film grain in compressed 

25 video. Film grain constitutes a high frequency quasi-random phenomenon that typically 
cannot undergo compression using conventional spatial and temporal methods that take 
advantage of redundancies in the video sequences. Attempts to process film-originated 
images using MPEG-2 or ITU-T/ISO H.264 compression techniques usually result in either an 
unacceptably low degree of compression or complete loss of the grain texture. 

30 Thus, there exists a need for a technique simulating film grain, especially a technique 

that affords relatively low complexity. 
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BRffiF SUMMARY OF THE INVENTION 

Briefly, there is provided a method in accordance with the present principles for 
simulating film grain. The method commences by creating a block (i.e., a matrix array) of 
transformed coefficients responsive to a band pass region associated with a desired grain 
pattern. (The band pass region is defined by a set of cut frequencies fm., fvu fm, and fvH that 
represent cut-off frequencies (in two dimensions) of a filter that characterizes the desired film 
grain pattern). The block of transformed coefficients undergoes an inverse transform to yield 
a bit-accurate film grain sample. If desired, the bit accurate sample can undergo scaling to 
enable blending with a video signal to simulate film grain in the signal. 

In practice, generation of the block of transformed coefficients occurs as follows. For 
each coefficient at coordinates (x,y) iri the block, where x and y are integers designating the 
row and column, respectively, a random value (r) is obtained, if each of the x and y 
coordinates lies within the band pass of the cut frequencies f^L and fnn and fvL and fvH, 
respectively. The value of the coefficient at the coordinates (x,y) becomes the random value 
r. The process is repeated for every entry in the block. 

The film grain simulation technique of the present principles affords reduced 
complexity as compared to prior- art transform-based approaches for film grain simulation by 
avoiding the computation of a direct transform. The technique also reduces the memory 
requirements of database-based approaches by storing a small set of transformed coefficients 
instead of film grain patterns. The present method has applicability to HD DVD systems, BD 
ROM systems as well as to satellite broadcasting, among other uses. 

DETAILED DESCRIPTION OF THE DRAWINGS 

FIGURE 1 depicts a block schematic diagram of the combination of a transmitter and 
receiver in a film Grain Processing chain useful for practicing the technique of the present 
principles; 

FIGURE 2 depicts, in flow chart form, the steps of a first method for creating a set of 
transformed coefficients for simulating film grain in accordance with the present principles; 

FIGURE 3 depicts, in flow chart form, the steps of a second method for creating a set 
of transformed coefficients for simulating film grain in accordance with the present principles; 
and 
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FIGURE 4 depicts, in flow chart form, the steps of a method for creating a database of 
film grain patterns in accordance with the present principles. 

DETAILED DESCRIPTION 

To best understand the technique of the present principles for simulating film grain 
using a set of cut frequencies that characterize the film grain pattern, a brief overview of film 
grain simulation will prove helpful. FIGURE 1 depicts a block schematic diagram of a 
transmitter 10, which receives an input video signal and, in turn, generates a compressed 
video stream at its output. In addition, the transmitter 10 also generates information indicative 
of the film grain (if any) present in the sample. In practice, the transmitter 10 could comprises 
part of a head-end array of a cable television system, or other such system that distributes 
compressed video to one or more downstream receivers 11, only one of which is shown in 
FIG. 1. The transmitter 10 could also take the form of encoder that presents media like 
DVDs. The receiver 1 1 decodes the coded video stream and simulates film grain in 
accordance with the film grain information and decoded video, both received from the 
transmitter 10 or directly from the media itself in the case of a DVD or the like, to yield an 
output video stream that has simulated film grain. The receiver 1 1 can take the form of a set- 
top box or other such mechanism that serves to decode compressed video and simulate film 
grain in that video. 

The overall management of film grain requires the transmitter 10 (i.e., the encoder) 
provide information with respect to the film grain in the incoming video. In other words, the 
transmitter 10 "models" the film grain. Further the receiver 1 1 (i.e., decoder) simulates the 
film grain according to the film grain information received from the transmitter 10. The 
transmitter 10 enhances the quality of the compressed video by enabling the receiver 1 1 to 
simulate film grain in the video signal when difficulty exists in retaining the film grain during 
the video coding process. 

In the illustrated embodiment of FIG. 1, the transmitter 10 includes a video encoder 12 
which encodes the video stream using any of the well known video compression techniques 
such as the ITU-T Rec. H.264 | ISO/EEC 14496-10 video compression standard. Optionally, a 
film grain remover 14, in the form of a filter or the like depicted in dashed lines in FIG. 1, 
could exist upstream of the encoder 12 to remove any film grain in the incoming video stream 
prior to encoding. To the extent that the incoming video contains no film grain, no need 
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would exist for the film grain remover 14. 

A film grain modeler 1 6 accepts the input video stream, as well as the output signal of 
the film grain remover 14 (when present). Using such input information, the film grain 
modeler 16 establishes the film grain in the incoming video signal. In its simplest form, the 
5 film grain modeler 16 could comprise a look up table containing film grain models for 

different film stocks. Information in the incoming video signal would specify the particular 
film stock used to originally record the image prior to conversion into a video signal, thus 
allowing the film grain modeler 16 to select the appropriate film grain model for such film 
stock. Alternatively, the film grain modeler 16 could comprise a processor or dedicated logic 

1 0 circuit that would execute one or more algorithms to sample the incoming video and 
determine the film grain pattern that is present. 

The receiver 1 1 typically includes a video decoder 18 that serves to decode the 
compressed video stream received from the transmitter 10. The structure of the decoder 18 
will depend on the type of compression performed by the encoder 12 within the transmitter 

15 10. Thus, for example, the use within the transmitter 10 of an encoder 12 that employs the 
ITU-T Rec. H.264 | ISO/IEC 14496-10 video compression standard to compress outgoing 
video will dictate the need for an H.264-compliant decoder 18. Within the receiver 1 1, a film 
grain simulator 20 receives the film grain information from the film grain model 16. The film 
grain simulator 20 can take the form of a programmed processor, or dedicated logic circuit 

20 having the capability of simulating film grain for combination via a combiner 22 with the 
decoded video stream. 

Film grain simulation aims to synthesize film grain samples that simulate the look of 
the original film content. As described, film grain modeling occurs at the transmitter 10 of 
FIG. I, whereas film grain simulation occurs at the receiver 1 1. In particular, film grain 

25 simulation occurs in the receiver 1 1 along with the decoding the incoming video stream from 
the transmitter 10 but upstream of the output of the decoded video stream. Note that the 
decoding process that occurs in the receiver 1 1 makes no use of images with added film grain. 
Rather, film grain simulation constitutes a post-processing method for synthesizing simulated 
film grain in the decoded images for display. For that reason, the ITU-T Rec. H.264 | 

30 ISO/EEC 14496-10 video compression standard contains no specifications regarding the film 
grain simulation process. However, film grain simulation requires information concerning the 
grain pattern in the incoming video signal, which information typically undergoes 
transmission in a Supplemental Enhancement Information (SEI) message when using the ITU- 
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T Rec. H.264 | ISO/IEC 14496-10 video compression standard as specified by the 
Amendment 1 (Fidelity Range Extensions) of that compression standard. 

In accordance with the present principles, there is provided a low-complexity method 
for simulating film grain, which presupposes that the transformation used to simulate film 
grain is linear. The method of the present principles, which is practiced by the film grain 
simulator 20 of FIG: 1, generates a block of film grain in a process that includes the following 
three steps: (1) create a block of transformed coefficients in accordance with the cut- 
frequencies for a desired film grain pattern, (b) compute an inverse transform and (c) scale the 
resultant values. A more complete description of each process follows hereinafter. 

Creating a block of transformed coefficients 

The process of creating of a block of transformed coefficients takes as its input a set of 
cut frequencies characterizing a desired film grain pattern and yields a set of normalized 
random numbers. In a preferred embodiment, the normalized random numbers will follow a 
normalized Gaussian random distribution (independent and identically distributed Gaussian 
random variable samples with zero mean and unity variance) while the cut frequencies are 
transmitted in a film grain characteristics SEI message, as defined by the ITU-T Rec. H.264 | 
ISO/IEC 14496-10 standard [1], 

In the illustrated embodiment, the creation of a block image B (e.g., a block of 
transformed coefficients of size N x N) in the frequency domain occurs in the manner 
illustrated in flow chart form in FIG. 2. The process begins at Start step 100 during which 
initialization occurs. Following step 100, step 102 occurs, commencing a loop which scans all 
coefficients of the N x N block by setting a pair of running variables (x,y) to all possible 
values in the integer range of [0,N-1] x [0,N-1]. The variables x and y each represent the x 
and y coordinates, respectively, of an entry in a block (i.e., a matrix array) of transformed 
coefficients produced by this process. During step 104, a check occurs whether the values of 
x and y lie within the band pass defined by the cut frequencies fHL and fm and fvL and fvn- • If 
so, step 106 occurs and a random number is generated and during step 108 the random 
number is assigned to the coefficient in the block at the x and y coordinates matching the 
current value of the running variables x and y, respectively. Step 104 illustrates a preferred 
embodiment of the checking of x and y values to decide whether or not they lie within the 
band pass defined by the cut frequencies, but other criteria are possible. 
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If the current values of x and y do not lie simultaneously within the band pass defined 
by the cut frequencies f HL and f H H and f V L and f V H of the desired film grain pattern, as 
transmitted in the SEI message, then the coefficient in the block associated with the current 
value of x and y is set to zero during step 1 10. Following either of steps 108 or 1 10, then step 
112 occurs, and the running variables x and y are incremented, provided that x and y < N and 
execution branches to step 104. Otherwise, the loop ends during step 1 12 and the method 

finishes at End step 114. 

The method for creating a block of transformed coefficients depicted in FIG. 2 can be 

described in a "for" computer program loop in the following manner: 

for(y = 0; y<N; y++) 

for(x = 0;x<N;x-H-) 

if((x<f H L&&y<fv L ) ||x>f H H II y>fvH) 
B[x][y] = 0 

else 

B[x][y] = r 

B[0][0] = 0 

Note that f H L and fvL constitute the horizontal and vertical low cut frequencies, respectively, 
fHH and fvH constitute the horizontal and vertical high cut frequencies, respectively of the 
desired film grain pattern, and r constitutes a normalized random number. In practice, a 
Gaussian random number r could be generated from two independent, uniformly distributed 
random values, using the Box-Muller transformation specified by: 

r = ^-21n(jcj) • sin(2«jr 2 ) 

where ln(.) is the natural logarithm, sin(.) is the sine function, 7i is Archimedes's constant, and 
xj and x 2 are in the range of (0,1), with 0 not included. 

FIGURE 3 depicts an alternate preferred method for creating a block of transformed 
coefficients. The flow chart method of FIG. 3 includes almost all of the same steps as the 
method of FIG. 2 and therefore like reference numbers appear in FIG. 3 as in FIG. 2 to 
describe like steps. The method of FIG. 3 differs in the following respect. Instead of 
obtaining a random number during step 106 in FIG. 2 through mathematical computation, the 
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method of creating a block of transformed coefficients of FIG. 3 includes the step 105 of 
obtaining a random number from a look-up table 1 10 of previously computed Gaussian 
random numbers. Otherwise, the two methods practice the same steps. 

Optionally, the block transformed coefficient generation methods of FIGS. 2 and 3 
could force the coefficient at the position 0,0 (i.e., the DC coefficient) to become zero in order 
to ensure that after inverse transform, the signal will have zero mean. Large deviations from 
zero for the DC coefficient could cause large deviations in the average value of the resulting 
film grain block. As a consequence, when blending the film grain with the decoded image, 
blocks could show visible variations in their average value (i.e., the blocks could become 
either darker or brighter). 

In the case when fai. and fvL both equal to zero, the creation of the block image B in 
the frequency domain can occur as follows: 

for(y = 0; y <N; y++) 

for( x = 0; x < N; x-h-) 

if(x>f HH || y>fvH) 
B[x][y] = 0 

else 

B[x][y] = r 

B[0][0] = 0 

Assuming also the ability to accomplish zero value initialization upon memory allocation of 
the block image, the previous method can be simplified to the following expression: 

for( y = 0; y < N; y++ ) 

for( x = 0; x < N; x++) 

if(x<=fHH && y<=fvH) 
B[x][y] = r 

Note that the processes described in FIGS. 2 and 3 make use of (f™ - fHi+l)*(fvH - fvL + 0 
normalized random values, thereby avoiding the filtering step that typically follows the 
computation of the forward transform of a block of normalized random.numbers. 
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Compute The Inverse Transform 

Following generation of the block of transformed coefficients, computation of the 
inverse transform occurs, typically according to the general formula: 

6 = a(C 7 xtfxC) 

where a constitutes a scaling factor that compensates for intensity variation resulting from the 
generation of the block of transformed coefficients. Typically, the scaling factor a can have a 
value of unity. 

When using a DCT transform, computation of the transformation matrix C occurs as 
follows: 



C= 



71/77 if m = 0 
Jl/N otherwise 



in this case, a = j—rr 



Scaling of the Inverse Transform Resultant Values 

Following the creation of the block of transformed coefficients and computation of the 
inverse transform of the block, the film grain simulation process of the present principles 
typically includes the step of scaling the film grain pattern to the appropriate film grain 
intensity, although such scaling not necessarily occur. For video coding applications, 
information about the film grain intensity typically undergoes transmission in the SEI 
message, as specified in the ITU-T Rec. H.264 | ISO/IEC 14496-10 standard. 
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The scaling process typically occurs in accordance in the following manner: 

for(y = 0;y<N; y++) 

for(x = 0;x<N;x++) 

b[ x ][ y ] *= scaling_factor 

This approach presents a tradeoff between complexity and memory requirements when 
compared to prior art methods. On one hand, the scaling method of the present principles 
reduces the complexity of transform-based approaches by avoiding computation of the direct 
transform and filtering in the frequency domain. On the other hand, the scaling method of the 
present principles reduces the memory requirements of database-based approaches by only 
storing a small set of transformed coefficients instead of complete film grain patterns. 

The method of the present principles can serve to initialize a database of film grain 
patterns. In particular, the steps of creating a the block of transformed coefficients and 
computing the inverse transform of the block could serve to initialize a database of film grain 
patterns as illustrated in flow chart form in Figure 4. The method of FIG. 4 commences upon 
execution of Start step 200 whereupon initialization occurs. Thereafter, step 202 occurs with 
generation of the block of Gaussian random values, using either the method of FIG. 2, or the 
method of FIG. 3. When employing the block transform generation method of FIG. 3, each 
random number entry is obtained by accessing the Gaussian random number look-up table 
107, as discussed previously. Following step 202, the block of Gaussian random values 
undergoes an inverse transform, typically via an Inverse Discrete Cosine Transform (BDCT) 
during step 204, along with the appropriate scaling. During step 206, the film grain patterns 
undergo storage in a database 207. Thereafter, the process ends at step 208. The process of 
FIG. 4 allows creation of the database 207 upon system initialization or reset, and enables 
storage in a conventional memory (usually SRAM memory). 

The foregoing describes a technique for simulating film grain in an image that affords 
the advantage of low complexity. * 
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CLAIMS 

1 1 . A method for simulating film grain, comprising the steps of: 

2 creating a block of transformed coefficients responsive to a band pass region 

3 associated with a desired film grain pattern; and 

4 establishing an inverse transform for the created block of transformed coefficients to 

5 yield a film grain sample. 

1 2. The method according to claim 1 wherein the block of block transformed 

2 coefficients is created by the steps of: 

3 (a) obtaining a value for each coefficient at coordinates (x, y) which is a random value 

4 for so long as x and y lie in the band pass defined by a set of cut frequencies f H L, fvu fiiH and 

5 fvH> but a zero otherwise; 

6 (b) setting the value of the coefficient at coordinates (x, y) to the obtained value; and 

7 (c) repeating steps (a) and (b) until a value is set for every coefficient in the block. 

1 3. The method according to claim 2 further comprising the step of setting the 

2 block coefficient at (0,0) to a zero value. 

1 4. The method according to claim 2 wherein each of the random values obtained for so 

2 long as x and y lie in the band pass defined by a set of cut frequencies f H u fvL, f™ and fvrt 

3 follows a Gaussian random distribution. 
1 

1 5. The method according to claim 2 wherein each of the random values is 

2 obtained by accessing an entry from a pre-computed look-up table of random values. 
1 

1 6. The method according to claim 1 further including the step of scaling the film 

2 grain sample. 
1 

1 7. A method of creating a block of transformed coefficients used to simulate a 

2 desired film grain pattern, comprising the steps of: 

3 (a) obtaining a value for each coefficient at coordinates (x, y) which is a random value 

4 for so long as x and y lie in a band pass region defined by the cut frequencies f H L, fvu 6ih and 
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5 fvH, but a zero otherwise; 

6 (b) setting the value of the coefficient at coordinates (x, y) to the obtained value; and 

7 (c) repeating steps (a) and (b) until a value is set for every coefficient in the block. 
1 

1 8. The method according to claim 7 further comprising the step of setting the 

2 block coefficient at (0,0) to a zero value. 

1 9. The method according to claim 7 wherein each of the random values obtained 

2 for so long as x and y lie in a band pass region defined by the cut frequencies f H L, fyL, froi and 

3 fvH follows a Gaussian random distribution. 

1 io. The method according to claim 7 wherein each of the random values is 

2 obtained by accessing an entry from a pre-computed look-up table of random values. 

1 11. The method according to claim 7 further including the step of establishing an 

2 inverse transform on the block of transformed coefficients. 

1 12. The method according to claim 1 1 further including the step of scaling the 

2 inverse transform of the block of transformed coefficients to yield a pattern of film grain 

3 samples. 

1 13. The method according to claim 12 further comprising the step of initializing a 

2 database of film grain patterns using a single block of transformed coefficients to create all 

3 possible film grain patterns. 

1 14. The method according to claim 12 further comprising the step of initializing a 

2 database of film grain patterns using multiple blocks of transformed coefficients to create all 

3 possible film grain patterns. 
1 

1 15. Apparatus for simulating film grain, comprising one of a programmed 

2 processor and logic circuit for (1) creating a block of transformed coefficients responsive to a 

3 band pass region associated with a desired film grain pattern; (2) establishing an inverse 

4 transform for the created block of transformed coefficients; and (3) scaling the block of 
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inversely transformed coefficients to create a simulated pattern of film grain. 

16. The apparatus according to claim 15 wherein the one of the programmed 
processor or logic circuit creates the block of block transformed coefficients by (a) obtaining a 
value for each coefficient at coordinates (x, y) which is a random value for so long as x and y 
lie in a band pass defined by the cut frequencies f H L, fvu ftiH and f V H, but a zero otherwise; (b) 
setting the value of the coefficient at coordinates (x, y) to the obtained value, and (c) repeating 
(a) and (c) until a value is set for every coefficient in the block. 

17. The apparatus according to claim 15 wherein the one of the programmed 
processor or logic circuit creates the block of block transformed coefficients by setting the 
block coefficient at (0,0) to a zero value. 

18. The apparatus according to claim 15 wherein the one of the programmed 
processor or logic circuit creates the block of block transformed coefficients such that each of 
the random values obtained for so long as x >= f H L or y >= f V L and x <= f^H and y <= fvH 
follows a Gaussian random distribution. 

► * 

19. The apparatus according to claim 1 5 wherein the one of the programmed 
processor or logic circuit obtains each of the random values by accessing an entry from a pre- 
computed look-up table of random values. 

20. Apparatus for simulating film grain, comprising: 

means for creating a block of transformed coefficients responsive to a band pass 
region associated with a desired film grain pattern; 

means for establishing an inverse transform for the created block of transformed 

coefficients; and 

means for scaling the block of transformed coefficients to create a simulated pattern of 
film grain. 
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